Assessing Transferability from Simulation to Reality for Reinforcement Learning

Learning robot control policies from physics simulations is of great interest to the robotics community as it may render the learning process faster, cheaper, and safer by alleviating the need for expensive real-world experiments. However, the direct transfer of learned behavior from simulation to reality is a major challenge. Optimizing a policy on a slightly faulty simulator can easily lead to the maximization of the `Simulation Optimization Bias` (SOB). In this case, the optimizer exploits modeling errors of the simulator such that the resulting behavior can potentially damage the robot. We tackle this challenge by applying domain randomization, i.e., randomizing the parameters of the physics simulations during learning. We propose an algorithm called Simulation-based Policy Optimization with Transferability Assessment (SPOTA) which uses an estimator of the SOB to formulate a stopping criterion for training. The introduced estimator quantifies the over-fitting to the set of domains experienced while training. Our experimental results on two different second order nonlinear systems show that the new simulation-based policy search algorithm is able to learn a control policy exclusively from a randomized simulator, which can be applied directly to real systems without any additional training

Underactuated Waypoint Trajectory Optimization for Light Painting Photography

Despite their abundance in robotics and nature, underactuated systems remain a challenge for control engineering. Trajectory optimization provides a generally applicable solution, however its efficiency strongly depends on the skill of the engineer to frame the problem in an optimizer-friendly way. This paper proposes a procedure that automates such problem reformulation for a class of tasks in which the desired trajectory is specified by a sequence of waypoints. The approach is based on introducing auxiliary optimization variables that represent waypoint activations. To validate the proposed method, a letter drawing task is set up where shapes traced by the tip of a rotary inverted pendulum are visualized using long exposure photography.Comment: Accepted for ICRA 2020 (International Conference on Robotics and Automation

Biological control of peach fungal pathogens by commercial products and indigenous yeasts.

The potential use of the commercial biocontrol products Serenade (Bacillus subtilis QST-713) and Trichodex (Trichoderma harzianum Rifai strain T39) to inhibit the postharvest pathogenic molds Penicillium crustosum and Mucor circinelloides was investigated. Both products exhibited antagonistic activity in vitro against the pathogens, reducing their growth at different levels. In addition, epiphytic yeasts isolated from peaches were identified as Candida maltosa, Pichia fermentans, and Pichia kluyveri by PCR-restriction fragment length polymorphism of internal transcribed spacer regions and screened for antagonistic activity against the same molds. The efficacy of biocontrol in vitro was dependent on the concentration of the yeast cells. Optimal yeast concentrations were above 10(7) CFU ml(-1). However, C. maltosa and P. fermentans were more effective than P. kluyveri in inhibiting molds. The exclusion of antifungal metabolite production and direct competition for nutrients or space with the pathogens was proposed as the mechanism of biocontrol. Application of biocontrol agents directly on artificially wounded peach fruits significantly reduced the incidence of mold rot during storage at 20 degrees C

Robot Learning From Randomized Simulations: A Review

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the “reality gap.” We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named “domain randomization” which is a method for learning from randomized simulations

Environmental monitoring and building simulation application to Vasari Corridor: Preliminary results

Abstract The Vasari Corridor has been used in the past and present for storage and presentation of works of art which require control of microclimate for optimal preservation. To this end, it was started the collaboration between the Uffizi Gallery and Laboratory of Environmental Physic of the Florence University for the environmental monitoring of microclimatic parameters, of which this work presents the preliminary results. It's was also created a three-dimensional model of the building in the stretch from the Uffizi Gallery to Ponte Vecchio, for the dynamic simulation of the energy behavior of the building validated by on-field measured values

Underactuated Waypoint Trajectory Optimization for Light Painting Photography

Despite their abundance in robotics and nature, underactuated systems remain a challenge for control engineering. Trajectory optimization provides a generally applicable solution, however its efficiency strongly depends on the skill of the engineer to frame the problem in an optimizer-friendly way. This paper proposes a procedure that automates such problem reformulation for a class of tasks in which the desired trajectory is specified by a sequence of waypoints. The approach is based on introducing auxiliary optimization variables that represent waypoint activations. To validate the proposed method, a letter drawing task is set up where shapes traced by the tip of a rotary inverted pendulum are visualized using long exposure photography

Minimally invasive vs. open segmental resection of the splenic flexure for cancer: a nationwide study of the Italian Society of Surgical Oncology-Colorectal Cancer Network (SICO-CNN)

Background Evidence on the efficacy of minimally invasive (MI) segmental resection of splenic flexure cancer (SFC) is not available, mostly due to the rarity of this tumor. This study aimed to determine the survival outcomes of MI and open treatment, and to investigate whether MI is noninferior to open procedure regarding short-term outcomes. Methods This nationwide retrospective cohort study included all consecutive SFC segmental resections performed in 30 referral centers between 2006 and 2016. The primary endpoint assessing efficacy was the overall survival (OS). The secondary endpoints included cancer-specific mortality (CSM), recurrence rate (RR), short-term clinical outcomes (a composite of Clavien-Dindo > 2 complications and 30-day mortality), and pathological outcomes (a composite of lymph nodes removed >= 12, and proximal and distal free resection margins length >= 5 cm). For these composites, a 6% noninferiority margin was chosen based on clinical relevance estimate. Results A total of 606 patients underwent either an open (208, 34.3%) or a MI (398, 65.7%) SFC segmental resection. At univariable analysis, OS and CSM were improved in the MI group (log-rank test p = 0.004 and Gray's tests p = 0.004, respectively), while recurrences were comparable (Gray's tests p = 0.434). Cox multivariable analysis did not support that OS and CSM were better in the MI group (p = 0.109 and p = 0.163, respectively). Successful pathological outcome, observed in 53.2% of open and 58.3% of MI resections, supported noninferiority (difference 5.1%; 1-sided 95%CI - 4.7% to infinity). Successful short-term clinical outcome was documented in 93.3% of Open and 93.0% of MI procedures, and supported noninferiority as well (difference - 0.3%; 1-sided 95%CI - 5.0% to infinity). Conclusions Among patients with SFC, the minimally invasive approach met the criterion for noninferiority for postoperative complications and pathological outcomes, and was found to provide results of OS, CSM, and RR comparable to those of open resection